Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tds experiment metrics #3839

Merged
merged 23 commits into from
Jan 23, 2025
Merged

Tds experiment metrics #3839

merged 23 commits into from
Jan 23, 2025

Conversation

SabrinaTardio
Copy link
Contributor

@SabrinaTardio SabrinaTardio commented Jan 21, 2025

Task/Issue URL: https://app.asana.com/0/1204186595873227/1208547600159201/f
Tech Design URL: https://app.asana.com/0/1204186595873227/1209001242859416/f

Description: Adds metric for TDS override experiment

Steps to test this PR:
Scenario 1: Not Enrolled in a TDS experiment

  1. Remove app
  2. Run the app check from the debugger that compilation pixel (m.content.blocking.compilation.loopsX.timeY) is sent with only the compilation time as parameter
  3. Try turn privacy on and off from menu and dashboard and check no experiment (experiment_metrics_tdsNextExperimentBaseline_treatment_ios_phone) or debug pixel (m.debug.breakage.experiment) is sent
  4. Try 3 quick refresh and check no experiment (experiment_metrics_tdsNextExperimentBaseline_treatment_ios_phone) or debug pixel (m.debug.breakage.experiment) is sent

Scenario 2: Enrolled in a TDS experiment

  1. Go to Settings -> All debug options -> Configuration URLs click on privacy config and override the URL with the following: https://staticcdn.duckduckgo.com/trackerblocking/config/test/macos-config.json
  2. check from the debugger that compilation pixel (m.content.blocking.compilation.loopsX.timeY) is sent with three parameters: compilation time, etag: 2ce60c57c3d384f986ccbe2c422aac44 and experimentName: “tdsNextExperimentBaseline_treatment"
  3. Try turn privacy on and off from menu and dashboard and check the following Pixels are sent: experiment_metrics_tdsNextExperimentBaseline_treatment_ios_phone ["pixelSource": "phone", "enrollmentDate": "2025-01-21", "conversionWindowDays": "0", "metric": "privacyToggleUsed", "value": "1”], Pixel fired m.debug.breakage.experiment.u ["tdsEtag": ""2ce60c57c3d384f986ccbe2c422aac44"", "enrolmentDate": "2025-01-21", "experiment": "tdsNextExperimentBaselinetreatment”]
  4. Check both are only sent once on privacy toggle
  5. Try 3 quick refresh and check 2x and 3x experiment pixel sent: experiment_metrics_tdsNextExperimentBaseline_treatment_ios_phone ["value": "1", "enrollmentDate": "2025-01-21", "metric": "2XRefresh", "conversionWindowDays": "0", "pixelSource": "phone"]
    experiment_metrics_tdsNextExperimentBaseline_treatment_ios_phone ["value": "1", "metric": "3XRefresh", "conversionWindowDays": "0", "enrollmentDate": "2025-01-21", "pixelSource": "phone”]
  6. check no extra m.debug.breakage.experiment.u is sent
  7. Remove the app reinstall it and follow step one again
  8. Try and 2 quick refresh of a page and check is sent m.debug.breakage.experiment.u ["tdsEtag": ""2ce60c57c3d384f986ccbe2c422aac44"", "enrolmentDate": "2025-01-21", "experiment": "tdsNextExperimentBaselinetreatment”]

<!—
Before submitting a PR, please ensure you have tested the combinations you expect the reviewer to test, then delete configurations you know do not need explicit testing.

Using a simulator where a physical device is unavailable is acceptable.
—>

Definition of Done (Internal Only):

Copy Testing:

  • Use of correct apostrophes in new copy, ie rather than

Orientation Testing:

  • Portrait
  • Landscape

Device Testing:

  • iPhone SE (1st Gen)
  • iPhone 8
  • iPhone X
  • iPhone 14 Pro
  • iPad

OS Testing:

  • iOS 15
  • iOS 16
  • iOS 17

Theme Testing:

  • Light theme
  • Dark theme

Internal references:

Software Engineering Expectations
Technical Design Template

@SabrinaTardio SabrinaTardio marked this pull request as ready for review January 21, 2025 14:11
Core/ContentBlocking.swift Outdated Show resolved Hide resolved
Core/ContentBlocking.swift Outdated Show resolved Hide resolved
@SabrinaTardio SabrinaTardio force-pushed the tds-experiment-metrics branch from cb047cc to ced22c6 Compare January 23, 2025 11:06
Copy link
Contributor

@jaceklyp jaceklyp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

SabrinaTardio added a commit to duckduckgo/BrowserServicesKit that referenced this pull request Jan 23, 2025
**Required**:

Task/Issue URL:
https://app.asana.com/0/1204186595873227/1208547600159201/f
iOS PR: duckduckgo/iOS#3839
macOS PR:  https://github.com/duckduckgo/macos-browser/pull/3755/files
What kind of version bump will this require?: Major

**Optional**:

Tech Design URL:
https://app.asana.com/0/1204186595873227/1209001242859416/f
CC:

**Description**: Adds pixel reporting for TDS override experiment
@SabrinaTardio SabrinaTardio merged commit fd448a4 into main Jan 23, 2025
13 checks passed
@SabrinaTardio SabrinaTardio deleted the tds-experiment-metrics branch January 23, 2025 14:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants